Device for monitoring the functioning of external synchronization modules in a multicomputer system

ABSTRACT

As soon as the computers of a two-out-of-three computer system arrive in their program run at a synchronization point, they interrupt the execution of the program run and transmit synchronization-readiness signals to the partner computers. The program run is continued at the same time in all computers, when synchronization-readiness signals from all three computers are present in these allocated synchronization modules. If a synchronization-readiness signal fails to appear, then this is detected by a time monitoring device of the synchronization module allocated to the computer which has become unsynchronized. After its established time span has expired, this synchronization module stops the computer from accessing the interface system relevant to security and transmits pseudo-synchronization-readiness signals to the other computers, which permanently simulate the synchronization capability of the failed computer there. The computer system then continues to work in the two-out-of-two mode. Should the time monitoring device fail, the time monitoring devices, which are adjusted to longer delay times, become effective in all synchronization modules and disconnect the computer system.

BACKGROUND OF THE INVENTION

The present invention relates generally to devices for monitoring thesynchronization in a multicomputer system having parallel-workingindividual computers, and more particularly to such a device thatevaluates the time between synchronization-readiness signals, which areproduced by the various computers of the multicomputer system or theirsynchronization modules and which signals indicate the attainment of aspecified synchronization point, and to such a device that severs acomputer, which does not output its synchronization-readiness signalwithin a specified time span after receiving thesynchronization-readiness signals from the other computers. Such adevice is disclosed by the German Provisional Patent 12 698 27.

Data flow among computers must be synchronized from time to time,especially in those multicomputer systems, in which the individualcomputers have separate clock-pulse generators and must exchange dataamong themselves. This is particularly necessary in situations where thenumber of computers is increased for security and reliability reasonsand, in some instances for availability reasons, and where the computerscontinually compare themselves to one another to check for conformity inorder to detect malfunctions. Computers also have to be synchronizedwhen data (e.g., messages and commands) are simultaneously input viaexternal interrupts. The maximum permissible time spans, within which amutual synchronization must take place, depend essentially upon theclock frequency of the computer-clock-pulse generators and upon theaccuracy of these clock-pulse generators.

German Provisional Patent 19 52 926 discloses a method for synchronizingtwo parallel-working data processing units, one of which is active andthe other of which constitutes a reserve unit. The active unit in eachcase generates synchronizing signals in periodic intervals. Thesesignals serve in the reserve unit to phase lock the clock-pulsegenerator there to the phase position of the clock signals from thecontrolling unit. This known method is not suited for multicomputersystems that have several controlling computers due to the unpredictablemanner in which the computer emitting the synchronizing signalsinfluences the data processing of the other computers, when correctingtheir clock-pulse generators.

German Provisional Patent 21 55 159 discloses an arrangement forsynchronizing a multitude of computers in a computer system, in whichthe individual computers are mutually synchronized by having thecomputer that is the first to reach a synchronization point anchored inits program transmit a synchronization signal via a common line sharedby all the computers to the remaining computers. This synchronizationsignal is stored in the remaining computers for a certain length oftime. It blocks the synchronization signals generated there within theremaining computers themselves, and activates a pulse-generatingcircuit, by means of which a counter for clock signals is forced intothe same switch position to which the corresponding counter of thefastest computer had been switched. This completes the synchronizationprocess. In the case of this known configuration for synchronizing thecomputers of a multicomputer system, the failure of one computer, orrather of the circuit elements allocated to this computer forsynchronization purposes, is not detected. Additionally, this knownconfiguration fails to verify whether or not the computer actuallyassumes the specified switch position in the slower-running computers.

German Provisional Patent 24 13 401 discloses a device for synchronizinga two-out-of-three computer system, in which the processing of a newcommand is made dependent upon at least two of the three computershaving established completion of the preceding command. Time-delayelements assure that the slowest computer at the time is able tocomplete the execution of commands and then simultaneously begin,together with the other computers, with the processing of the followingcommand. If the slowest computer is not able to do this, it falls out ofstep, and is unable to synchronize itself. The computer system as suchremains operational then as a two-out-of-two system. This device isunable to determine that one of the computers has failed because it isunable to be synchronized with the other computers; hence, notroubleshooting operation is launched. Thus, the failure of a secondcomputer causes the computer system to become non-operational.

German Provisional Patent 12 698 27 discloses a method and a device forsynchronizing two parallel-working data processing systems, in which thesynchronization signals generated by the two data-processing devices aremonitored in a timer supervision routine to verify that they do not runtoo far apart. If they are running apart to an unacceptable degree,however, then a program interrupt occurs due to a timing error. If thetwo synchronization signals exist within the maximum time durationspecified by the timer supervision, then they initiate a synchronizationroutine in both individual computers through an AND operation. Since thetimer supervision is not supposed to respond during the runningoperation, its performance must be verified by test programs to ensurethat it is actually effective in case of a malfunction as well. Thesetest programs adversely affect the application programs running in thedata-processing devices and slow down the effective operating speed ofthe computer system.

German Published Patent Application 34 31 169 discloses a method forsynchronizing several parallel-working computers, in which each computerinterrupts its program in response to a signal received from anothercomputer indicating its synchronization readiness, and when theconditions are present for its part, it outputs a corresponding signalto all the other computers. Each computer begins with the processing ofthe next program step, after all computers of the computer system havesignalled their synchronization readiness. Therefore, in this case, theprocessing speed of the fastest computer is adapted to that of theslowest computer of the multicomputer system. To prevent the situationfrom occurring in which the entire multicomputer system can no longercontinue functioning after one computer fails, the computers alsocontinue with their program when, in addition to their ownsynchronization-readiness signal, the corresponding signal from anothercomputer is also available, and a certain specified minimum time haselapsed. However, no means are available for detecting and disconnectingan individual computer that has become out of synchronization with theother computers. In particular, this known device does not disclose, incase of a malfunction, means for severing the computer that is no longerreliably operational from the multicomputer system, in which the meansfor severing operate within the still operational computers.

The present invention is directed to the problem of developing a devicefor monitoring the synchronization in a multicomputer system consistingof parallel-working individual computers by evaluating the delay betweensynchronization-readiness signals that are produced by the variouscomputers of the multicomputer system or their synchronization modulesand that indicate the computer has reached a specified synchronizationpoint. The present invention is also directed to the problem ofdeveloping such a device that severs a computer that does not output itssynchronization-readiness signal within a specified time span after thedevice receives the synchronization-readiness signals from the othercomputers, and which device guarantees that a defective computer can bereliably detected and severed from the computer system, when thedefective computer is unable to be synchronized with the remainingcomputers. The present invention is also directed to the problem ofdeveloping a device that performs the above stated functions withoutinterrupting the performance of the application program in anundesirable manner. Finally, the present invention is directed to theproblem of developing such a device in which when a single computerbecomes unsynchronized, the remaining computers continue to operate inthe two-out-of-two mode, and in which only when the unsynchronizedcomputer is unable to be easily severed, is the computer systemaltogether disconnected for security reasons.

SUMMARY OF THE INVENTION

The present invention solves these problems by allocating to eachcomputer a hardware-synchronization module, which is capable of beingcontrolled by the clock signals of an internal clock-pulse generator,and which outputs the synchronization-readiness signals. The presentinvention also provides a first time monitoring device for eachsynchronization module. This first time monitoring device is capable ofbeing controlled by means of the synchronization-readiness signalsoutput from the synchronization modules allocated to the othercomputers. Upon expiration of the specified time span, the first timemonitoring device directs a fault-storage means to disconnect at leastthose output gates of the allocated computer that are relevant tosecurity, and to inject pseudo-synchronization-readiness signals on thecommon lines through which the other computers of the multicomputersystem or rather their synchronization modules are informed of thesynchronization readiness of the computer in question. The presentinvention also provides each synchronization module with two timemonitoring devices, whose delay time is greater than that of the firsttime monitoring device. These two time monitoring devices arecontrollable by the corresponding synchronization module when asynchronization-readiness signal is emitted, as well as when thesynchronization-readiness signals or thepseudo-synchronization-readiness signals from the other synchronizationmodules are received. Upon expiration of the delay time of these twotime monitoring devices, which occurs independently of one another,these two time monitoring devices disconnect at least the output gatesof the corresponding computer and of the synchronization module that arerelevant to security.

By applying redundant time monitoring devices (or timeout facilities) todisconnect all the individual computers of the multicomputer system,computers that are unable to be synchronized can be reliablydisconnected. An additional time monitoring device set to a shortermonitoring time ensures that, if a single computer fails, the other timemonitoring devices of the other computers do not function in response toan actuation, and that the computer system can continue to functionwhile excluding the defective computer. This is achieved in that thisfurther time monitoring device compulsorily switches the output signalsfrom the failed individual computer into the "ready for synchronization"setting. When the defective time monitoring device fails, the redundanttime monitoring device of the other computers causes the computer systemto be reliably disconnected.

To guarantee that the computer system is reliably disconnected in caseof a malfunction, the present invention provides for the redundant timemonitoring device to act in different ways on the process, whennecessary, so that even when the circuit element controlled by one ofthese time monitoring devices falls, the process can still be reliablyaffected.

Adjusting the time monitoring devices, by means of which a computer issupposed to be severed from the computer system, when necessary, is madedependent in accordance with the teaching of the present invention uponthe existence of appropriate control signals from the still operationalcomputers, or rather from the synchronization modules allocated to thesecomputers. This ensures that, even when one computer is completelyinoperative, the computers that are still operating can reliably cut offthe failed computer.

The present invention also provides that the control signals, which comefrom the other computers and which adjust the time monitoring devices inthe synchronization modules, are fed via OR elements. These OR elementscan be switched into an active state, when needed, by the correspondingcomputer. When the computer activates the OR elements, as soon as thecomputer receives an indication that another computer has failed in themulticomputer system, it is then possible in accordance with the presentinvention for the defective computer to be severed from themulticomputer system so that it can be serviced. This can be donewithout creating interfering signals on the link circuits thatunintentionally adjust the time monitoring devices in thesynchronization modules of the still intact computers.

According to the teaching of the present invention, the design of thecontrol outputs of the synchronization module must be such that they aredecoupled from the control inputs of the other synchronization modules,to rule out unintentional conducted interference.

To simplify the service measures, the present invention provides forarranging means for optically characterizing the operating states of atleast a few of the discrete components on the synchronization modules.Thus, for example, the response of the time monitoring devices or theadjustment of the fault storage means can be marked there.

The computer of the multicomputer system is synchronized often enoughwhen, according to the teaching of the present invention, specificcontrol signals, which are being applied to the computer buses, initiatethe synchronization processes.

To prevent the multicomputer system from being disconnected upon initialstart up or upon re-start because it is unable to be synchronized withinthe specified times, the present invention provides for the timemonitoring devices to be switched into an inactive state for a certaintime.

BRIEF DESCRIPTION OF THE DRAWING

The single drawing shows a secure two-out-of-three computer systemconsisting of three computers MCA through MCC, whose individualcomputers have identical hardware and process identical data in parallelusing identical software, which computer system operates according tothe present invention.

DETAILED DESCRIPTION

In a manner not shown in the figure, three computers MCA, MCB and MCCcontinually compare data being applied to their buses to check forconformity via monitoring channels of a dual design running among thecomputers. When the data of one computer deviates from that of theremaining computers, the computer detects this deviation from the datacomparison and uncouples itself from the computer system by severing itsconnection, at least to the process interface system that is relevant tosecurity. In addition this connection is also interrupted via themonitoring channels. The secure multicomputer system, which up to thispoint has been working as a two-out-of-three system, then continues towork as a two-out-of-two system, reliably from a standpoint of signalengineering. This continues until the failed computer is repaired andphased back into the multicomputer system. The drawing does not depictthe circuit elements, through which the process interface system to becontrolled by the multicomputer system receives signals from thecomputers.

Each of the individual computers MCA through MCC has its own clock-pulsegenerator (not shown), which supplies clock signals for data processing,data transmission, and data comparison. These clock-pulse generators arepreferably quartz-crystal-controlled; their clock frequency lies in theorder of magnitude of, for example, 10 MHz, and the quartz accuracy atabout 10⁻⁴ /s. This means that the clock pulses of the three clock-pulsegenerators can deviate from one another every second by up to 1000 Hz.It follows from this that the three computers, although they process thesame controlling tasks according to the same program, they do notnecessarily do this at exactly the same time, but rather more or less instaggered intervals. A comparison to check for conformity then turns outto be negative. Problems also occur when data are phased into thecomputers, and then the computers read in these data at differentclock-pulse steps of their program. Thus one computer could already bereading in the data, while another is still waiting to do this. Thiswould mean that different data are being applied--even if onlytemporarily--to the computers. To guarantee a sufficiently precisesynchronization of the computers when reading in the data and during thedata comparison, it is necessary for the three computers to becontinually synchronized by means of WAIT cycles and, in fact, given theaccepted parameters, within a time interval of less than 1 ms. In thecase of one concrete refinement of the multicomputer system, one startsout from a reciprocal synchronization in the order of magnitude of about10 μs. The synchronization process is initiated by the individualcomputers of the computer system when peripheral data are read in andout and when certain memory-read cycles are performed. These are thecontrol signals IOW, IOR and MEMS, whereby only every second or thirdmemory-read signal MEMS is supposed to initiate the synchronizationprocess. The rate of occurrence of these control signals is such thatthe desired synchronization cycle time of about 10 μs is achieved. As aresult of the synchronization process, the program processing of thevarious computers rum two clock pulses apart from one another, at themost. This slight offset of the program processing allows the dataexisting on the internal computer buses to be mutually compared forconformity, in each case during the synchronization phase.

The three computers of the multicomputer system are synchronized viahardware-synchronization modules BGA through BGC situated outside of thecomputers. These hardware-synchronization modules are designed so thatin case of need, namely when a computer is unable to be synchronizedwith the remaining computers, these hardware-synchronization modulescause the computer that has become unsynchronized to be severed from thecomputer system. This severing operation is initiated by thesynchronized computers and the defective computer is unable to influencethe severing operation, i.e., the defective computer cannot preventitself from being cut off. Severing a computer does not necessarily haveto be occur by disconnecting the computer by interrupting its currentsupply. Rather, in case of a malfunction, it is sufficient to disconnectonly those output gates of the defective computer that are relevant tosecurity, so that this computer is unable to cause danger. At the sametime, test programs can attempt to discover the reason for themalfunction that has occurred, so that repair of the defective computercan begin as soon as possible.

When the computers arrive in the program processing at a synchronizationpoint defined by the previously mentioned control signals, then ahardware control StA, StB or StC temporarily halts further execution ofthe program on the synchronization modules BGA through BGC by notsetting READY signals, whereby the last command to be executed is quasifrozen. The hardware control StA, StB or StC via output amplifiers VA,VB or VC causes synchronization-readiness signals SYA, SYB or SYC to beinjected on the link circuits leading to the other synchronizationmodules. The synchronization-readiness signals from the correspondingpartner computers are gated in pairs in the synchronization modules ofthe computers and in AND gates UA, UB, or UC, and cause hardware timemonitoring devices ZUA1 to ZUA3, ZUB1 to ZUB3, or ZUC1 to ZUC3 to beadjusted there. Each synchronization module evaluates thesynchronization-readiness signals being received and causes the programprocessing to be continued by injecting a READY signal, when thesynchronization-readiness signals from all three computers are received.This occurs simultaneously in the case of all synchronization modulesand is triggered by the slowest clock-pulse generator of the computersto be synchronized. However, if this clock-pulse generator is so slowthat a synchronization process is unable to be performed within aspecified maximum time or if, because of a malfunction, one of thecomputers processes a different program sequence than the othercomputers, then the synchronization modules of these computers cause theunsynchronized computer to be severed from the computer system. To thisend, the synchronization-readiness signals from the still intactcomputers adjust, inter alia, the time monitoring device ZUA3 in thesynchronization module of the computer to be severed, for instance ofthe computer MCA. The delay time of this time monitoring device isselected to correspond to the maximum permissible time delay of theprogram processing of this computer compared to those of the othercomputers. In case the computer MCA is unable to be synchronized, thetime monitoring device ZUA3 adjusts a fault storage means FSA, whichemits a disconnect signal ASA for severing from the computer MCA theprocess interface system, which is relevant to security. The timemonitoring device ZUA3 makes this adjustment after the expiration of aperiod of time of, for example, 3 μs, that is about 30 cycle times fromthe time that the synchronization readiness is signalled by the othersynchronization modules. At the same time, the synchronization moduleBGA causes the pseudo-synchronization-readiness signals SYA, which aresimulating the synchronization readiness of the disconnected computerMCA, to be injected on to the link circuits leading to the othersynchronization modules. Thus, the computer system can continue to bereliably operated from a standpoint of signal engineering in thetwo-out-of-two mode after the failure of the computer MCA. Thepseudo-synchronization-readiness signals output by the synchronizationmodule of the failed computer allow the synchronization process in thesynchronization modules of the controlling computers to operateindependently of the synchronization module allocated to the failedcomputer, i.e., the synchronization module allocated to the failedcomputer no longer participates in the synchronization of the remainingsynchronization modules. In the case of a failed computer MCA, the inputof the AND gate UB at the top of the drawing and the input of the ANDgate UC at the bottom of the drawing are constantly influenced by thepseudo-synchronization-readiness signal SYA coming from thissynchronization module, so that the AND gate in question is activatedwhen the synchronization-readiness signal is available from the partnercomputer that is still available at the time, and it adjusts the timemonitoring device in the two synchronization modules BGB and BGC.Corresponding processes follow when the computers MCB or MCC are unableto be synchronized, whereby the corresponding synchronization modulesgenerate corresponding disconnect signals ASB or ASC for the outputgates leading to the process interface system.

Usually, only very few clock signals are needed to synchronize thecomputers of a multicomputer system, so that, altogether, only a smallpercentage of the available computer time is needed for the ongoingsynchronization. One of the time monitoring devices ZUA3 to ZUC3responds only by way of exception and causes the corresponding computerto be disconnected.

To ensure that, in case of need, these time monitoring device actuallyreact in a manner appropriate to their tasks, one could subject them toa performance check through test programs. However, this would make itnecessary to interrupt the application program running at the time andwould cost computer time. For this reason, in addition to the first timemonitoring device ZUA3 to ZUC3, the synchronization modules haveadditional time monitoring devices ZUA2 to ZUC2, which are adjusted,together with the first time monitoring device, by means of thecorresponding control. The delay time of these additional timemonitoring devices is greater than that of the first time monitoringdevice. Thus, assuming proper functioning performance, it is ensuredthat a first time monitoring device ZUA3 can respond, before theadditional time monitoring device ZUB1, ZUB2, ZUC1, ZUC2 of the othercomputers respond. The multicomputer system would be altogetherdisconnected when this time monitoring device responds. These timemonitoring devices had been previously adjusted when thesynchronization-readiness signal was emitted by the synchronizationmodule in question.

For reasons of security, two time monitoring devices ZUA1, ZUA2 or ZUB1,ZUB2 or ZUC1, ZUC2 are provided for each synchronization module. Whenactuated, these time monitoring devices block the output gates of thecomputer that are relevant to security and are affected by themalfunction, independently of one another and in different ways. Whilethe time monitoring device ZUA2 of the synchronization module BGAoutputs a disconnect signal ASA via the output of the fault storagemeans FSA for the output gates of the process interface system that arerelevant to security, and via the output of the computer MCA, by way ofthe control StA, the time monitoring device ZUA1 causes thesynchronization stop to be abandoned and the output gates leading to thecomputer interface system to be disconnected by means of the computerMCA as the result of an interrupt NMI, which is unable to be masked, tothe computer MCA. If a synchronization process occurs after the timemonitoring devices are adjusted within their established delay times,then the corresponding control causes the time monitoring devices to bereset and initiates the program start by way of the READY control.

While the additional time monitoring devices are present exclusively forreasons of security and limit the waiting time of the computers to onesynchronization process, as well as act as a fault detector, for thecorresponding first time monitoring devices, the first time monitoringdevices increase the reliability of operation of the computer system.This is because they are supposed to become effective when one computeris unable to be synchronized and, by means of thepseudo-synchronization-readiness signals, they make it possible for theprogram processing to be continued in the remaining two-out-of-twocomputer systems.

To facilitate the servicing of the computer system, it is advantageousto provide the synchronization modules with means for opticallycharacterizing the operating states of at least a few of theircomponents. Thus, for example, from the characterization of theoperating state of the time monitoring devices or of the fault-storagemeans, a statement can be made about which computer was cut off andthrough which means it was cut off. The serviceability of the computersystem can still be further improved by having the computers generatecontrol signals SA1, SA2 or SB1, SB2 or SC1, SC2 each time aninoperative computer is detected. Via corresponding OR gates OA1, OA2 orOB1, OB2 or OC1, OC2, these control signals mask those inputs of theirAND gates UA through UC, to which the synchronization-readiness signalor pseudo-synchronization-readiness signal coming from the failedcomputer must be fed. This prevents synchronization attempts from beingunintentionally initiated by interference effects on the open inputs ofthe AND gate, when the modules of one computer are withdrawn or when asynchronization module is withdrawn. In some instances, suchunintentional synchronization attempts could cause the entire computersystem to be disconnected. Thus, the individual computers generate thepseudo-synchronization signals coming from the synchronization module ofthe failed computer, by themselves, and, in this manner, make themselvescompletely independent from further participation of this computer, orrather of its synchronization module. To restart the computer system inthe two-out-of-three mode, the control signals of the computers whichswitch through the AND gates must, of course, be reset again. To thisend, a software synchronization must initially follow, which allows theprograms of the individual computers to be pre-synchronized to anaccuracy of a few ten program cycles and to subsequently undergo aprecision synchronization through the synchronization modules, asclarified in detail above.

I claim:
 1. An apparatus for monitoring synchronization of a pluralityof parallel-working individual computers in a multicomputer system, inwhich each of said computers has a plurality of security relevant outputgates coupled to a device that might be impacted by a malfunction insaid each computer said apparatus comprising:a) a plurality of lines;and b) a plurality of synchronization modules being coupled together bysaid plurality of lines, one such module being provided for each of saidplurality of computers, each synchronization module outputting asynchronization-readiness signal to other synchronization modules ofsaid plurality of synchronization modules via said plurality of lines,wherein each synchronization module outputs thesynchronization-readiness signal when a specifiable synchronizationpoint has been reached in an associated computer of the plurality ofparallel-working individual computers, which associated computer isassociated with said each synchronization module, each of saidsynchronization modules including:(i) an internal clock-pulse generator;(ii) a first time monitoring device being responsive to a firstplurality of synchronization-readiness signals from others of saidplurality of synchronization modules, and detecting whether theassociated computer produces a control signal within a firstpredetermined time measured from when the first time monitoring devicereceives a synchronization-readiness signal from one of the othersynchronization modules; (iii) a fault-storage means being set by saidcontrol signal, disconnecting at least those security relevant outputgates of the associated computer and injecting a pseudosynchronization-readiness signal on the plurality of lines to the othersynchronization modules, if said first predetermined time expires; (iv)a second time monitoring device being responsive to the control signalfrom the associated computer and the synchronization-readiness signalsof the other synchronization modules, and causing the security relevantoutput gates of the associated computer to be disabled after a secondpredetermined time expires, wherein said second predetermined time isgreater than said first predetermined time; and(v) a third timemonitoring device being responsive to the control signal from theassociated computer and the synchronization-readiness signals of theother synchronization modules, and said third time monitoring devicedisabling the security relevant output gates of the associated computerafter a third predetermined time expires, wherein said third timemonitoring device operates independently of said second time monitoringdevice, and said third predetermined time is greater than said firstpredetermined time.
 2. The apparatus according to claim 1, furthercomprising first means for disabling the security relevant output gatesbeing controlled by said second time monitoring device, and second meansfor disabling the security relevant output gates being controlled by thethird time monitoring device.
 3. The apparatus according to claim 1,wherein said first time monitoring device further comprises a controlinput, said second time monitoring device further comprises a controlinput, said third time monitoring device further comprises a controlinput, and each synchronization module further comprises an AND gate,said AND gate having a first plurality of inputs receiving the firstplurality of synchronization-readiness signals and the first pluralityof pseudo-synchronization-readiness signals of the other synchronizationmodules, and having an output being coupled to the control inputs ofsaid first, second and third time monitoring devices.
 4. The apparatusaccording to claim 3, wherein said AND gate further comprises a secondplurality of inputs, each synchronization module further comprises aplurality of OR gates having a plurality of outputs coupled to thesecond plurality of inputs of the AND gate, each of said plurality of ORgates having a first input receiving the first plurality ofsynchronization-readiness signals or the first plurality ofpseudo-synchronization-readiness signals of the other synchronizationunits respectively, and each of said plurality of OR gates having asecond input receiving a control signal being generated by theassociated computer.
 5. The apparatus according to claim 1, wherein eachof said synchronization modules further comprises a control input and acontrol output, wherein the control output of the synchronization moduleis decoupled from the control inputs of the other synchronizationmodules.
 6. The apparatus according to claim 1, wherein each of saidplurality of synchronization modules further comprises means foroptically characterizing a current operating state of thatsynchronization module by switching on and off light emitting elements,wherein the operating state includes information regarding discretecomponents of that synchronization module.
 7. The apparatus according toclaim 1, wherein when the multicomputer system is started or restarted,functioning of the first, second and third time monitoring devices isinhibited for a specifiable number of synchronization cycles bygenerating the pseudo-synchronization-readiness signals.
 8. Amulticomputer system comprising:a) a plurality of parallel-workingindividual computers, each of said plurality of parallel-workingindividual computers having a plurality of security relevant gates thatare coupled to a device that might be impacted by a malfunctioning saideach computer; b) an apparatus for monitoring synchronization of saidplurality of parallel-working individual computers, said apparatusincluding:(i) a plurality of lines; and (ii) a plurality ofsynchronization modules being coupled together by said plurality oflines, one such module being provided for each of said plurality ofcomputers, each synchronization module outputting asynchronization-readiness signal to other synchronization modules ofsaid plurality of synchronization modules via said plurality of lines,wherein each synchronization module outputs thesynchronization-readiness signal when a specifiable synchronizationpoint has been reached in an associated computer of the plurality ofparallel-working individual computers, which associated computer isassociated with said each synchronization module, each of saidsynchronization modules including: (a) an internal clock-pulsegenerator; (b) a first time monitoring device being responsive to afirst plurality of synchronization-readiness signals from others of saidplurality of synchronization modules, and detecting whether theassociated computer produces a control signal within a firstpredetermined time measured from when the first time monitoring devicereceives a synchronization-readiness signal from one of the othersynchronization modules; (c) a fault-storage means being set by saidcontrol signal, disconnecting at least the security relevant outputgates of the associated computer and injecting a pseudosynchronization-readiness signal on the plurality of lines to the othersynchronization modules, if said first predetermined time expires; (d) asecond time monitoring device being responsive to the control signalfrom the associated computer and the synchronization-readiness signalsof the other synchronization modules, and disabling the securityrelevant output gates of the associated computer after a secondpredetermined time expires, wherein said second predetermined time isgreater than said first predetermined time; and (e) a third timemonitoring device being responsive to the control signal from theassociated computer and the synchronization-readiness signals of theother synchronization modules, and said third time monitoring devicedisabling the security relevant output gates of the associated computerafter a third predetermined time expires, wherein said third timemonitoring device operates independently of said second time monitoringdevice, and said third predetermined time is greater than said firstpredetermined time.
 9. The multicomputer system according to claim 8,wherein the first time monitoring, device has a control input, thesecond time monitoring device has a control input, the third timemonitoring device has a control input, each of said plurality ofsynchronization modules further comprises an AND gate, said AND gatehaving a first plurality of inputs receiving the first plurality ofsynchronization-readiness signals or the first plurality ofpseudo-synchronization-readiness signals of the other synchronizationmodules, having an output being coupled to the control inputs of saidfirst, second and third time monitoring devices, and having a secondplurality of inputs.
 10. The multicomputer system according to claim 9,wherein each of said AND gates further comprises a second plurality ofinputs, each of said plurality of synchronization modules furthercomprises a plurality of OR gates having outputs coupled to the secondplurality of inputs of the AND gate, each of said plurality of OR gateshaving a first input receiving one of the first plurality ofsynchronization-readiness signals or one of the first plurality ofpseudo-synchronization-readiness signals of the other synchronizationmodules, and each of said plurality of OR gates having a second inputreceiving a control signal being injected by the associated computer,whereby the associated computer detects when a second computer of theplurality of computers has failed by performing a data comparison and,as a result, acts upon said plurality of OR gates that read in aparticular synchronization-readiness signal from a particularsynchronization module of said plurality of synchronization modules thatis allocated to said second computer so that said particularsynchronization-readiness signal is inhibited from affecting the outputof said AND gate, until said second computer is again integrated intothe multicomputer system.
 11. The multicomputer system according toclaim 8, wherein each of said plurality of synchronization modulesfurther comprises means for optically characterizing a current operatingstate of that synchronization module by switching on and off lightemitting elements, wherein the operating state includes informationregarding discrete components of that synchronization module.
 12. Themulticomputer system according to claim 8, wherein when themulticomputer system is started or restarted, functioning of the first,second and third time monitoring devices is inhibited for a specifiablenumber of synchronization cycles by generating thepseudo-synchronization-readiness signals.
 13. A method for synchronizinga plurality of parallel-working individual computers in a multicomputersystem, in which each of the plurality of parallel-working individualcomputers has a plurality of security relevant output gates coupled to adevice that might be impacted by a malfunction in said each computersaid method comprising the steps of:a) allocating a synchronizationmodule to each computer of the plurality of parallel-working individualcomputers; b) outputting a first synchronization-readiness signal from afirst synchronization module allocated to a first computer of theplurality of computers to other synchronization modules that areallocated to others of the plurality of computers upon receipt of afirst control signal from the first computer, wherein the first controlsignal, and thus the synchronization-readiness signal, indicate that aspecifiable synchronization point has been reached in said firstcomputer; c) disconnecting at least the plurality of security relevantoutput gates of the first computer by using a first timeout device whena first predetermined time expires in which a first plurality ofsynchronization-readiness signals from the other synchronization modulesare received by the first synchronization module but the first controlsignal from the first computer is not received by the firstsynchronization module; and d) injecting apseudo-synchronization-readiness signal from the first synchronizationmodule on lines interconnecting the other synchronization modules, ifthe first predetermined time expires.
 14. The method according to claim13, further comprising the steps of:e) disconnecting the plurality ofsecurity relevant output gates of the first computer with a secondtimeout device when a second predetermined time expires that is greaterthan the first predetermined time; f) controlling a control input of thesecond timeout device with the synchronization-readiness signal emittedfrom that synchronization module allocated to the first computer, whenthe first plurality of synchronization-readiness signals are received bythat synchronization module or when the first plurality ofpseudo-synchronization-readiness signals from said other synchronizationmodules are received by that synchronization module; g) disconnectingthe security relevant output gates with a third timeout device when athird predetermined time expires that is greater than the first delaytime; and h) controlling a control input of the third timeout devicewith the synchronization-readiness signal emitted from thatsynchronization module allocated to the first computer, when the firstplurality of synchronization-readiness signals are received by thatsynchronization module or when the first plurality ofpseudo-synchronization-readiness signals from the other synchronizationmodules are received by that synchronization module.
 15. The methodaccording to claim 14, further comprising the step of:i) controlling thecontrol inputs of the second and third timeout devices with the outputof an AND gate, wherein the AND gate has a first input receiving thefirst plurality of synchronization-readiness signals or the firstplurality of pseudo-synchronization-readiness signals.
 16. The methodaccording to claim 15, further comprising the steps of:j) controlling aplurality of inputs of the AND gate with a plurality of OR gatescorresponding in number to one less than a number of the synchronizationmodules, wherein each of the plurality of OR gates have a first inputreceiving one of the first plurality of synchronization-readinesssignals or one of the first plurality ofpseudo-synchronization-readiness signals, and each of the plurality ofOR gates having a second input receiving a control signal being injectedby the allocated computer; and k) inhibiting a particularsynchronization-readiness signal from a particular synchronizationmodule allocated to a second computer of the plurality ofparallel-working individual computers from affecting the output of theAND gate in that synchronization module allocated to the first computer,when the first computer detects that the second computer has failed byperforming a data comparison and then acting upon said plurality of ORgates that read in the particular synchronization-readiness signal,until the second computer is again integrated into the multicomputersystem.
 17. The method according to claim 13, further comprising thestep of: inhibiting functioning of the first, second and third timeoutdevices for a specifiable number of synchronization cycles when themulticomputer system is started or restarted, after generating thepseudo-synchronization-readiness signal.